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Effects of Check & Connect on Attendance, Behavior, and Academics: 
A Randomized Effectiveness Trial 


Background / Context: 

Although graduation rates in the US have improved from 51.3% in 1972 to 89.6% in 
2010, dropout remains a serious social and economic issue, particularly among low- income 
students and non- white student subgroups (Chapman, Laird, & KewalRamani, 2010). The causes 
of dropout are varied and complex but several distinct and malleable factors have been identified 
Four of the most salient and pliable student-level factors linked to dropout are academic 
achievement, attendance, engagement, and behavior (Allensworth & Easton, 2007; Balfanz, 
Herzog, & Mac Iver, 2007; Henry, Knight, & Thornberry, 2012; Kurlaender, Reardon, & 
Jackson, 2008; Ou, Mersky, Reynolds, & Kohler, 2007; Rumberger & Lim, 2008; Silver, 
Suanders, & Zarate, 2008). As such, many dropout prevention and intervention programs target 
these risk factors. 

Communities In Schools (CIS; www.communitiesinschools.org) is one of the largest and 
most widely disseminated dropout prevention programs that specifically target academic 
achievement, attendance, engagement, and behavior. CIS is a nonprofit, nationwide network of 
nearly 200 independent affiliates delivering a dropout prevention and intervention model in more 
than 3,000 schools across 28 states. The CIS model uses school-based case managers to develop 
community partnerships, bring local resources to school campuses, and provide direct services to 
schools and students at risk of dropping out. Although CIS outlines a specific process for service 
delivery that targets the four predictors of dropout, the CIS model does not specifically prescribe 
interventions. That is, local affiliates utilize local data to select empirically-based interventions 
that fit local school and student needs. One local CIS affiliate in Texas was challenged with a 
large group of non-white students who exhibited school attendance problems — one of the four 
risk factors accounting for the largest proportion of variance in dropout. 

To address the challenges CIS was experiencing with absentee students, CIS staff 
partnered with a university research group to implement a field-based trial of Check & Connect 
(C&C; Sinclair et al., 1998). C&C is a widely-used intervention for improving school attendance 
and reducing dropout; however, the evidence base for C&C largely relies on less than optimal 
studies implemented by the program developers. The study reported here implemented C&C 
within a CIS service delivery model. That is, all participating CIS schools implemented 
universal, school-wide interventions and individualized case management services to improve 
academics, behavior, and attendance. However, within participating CIS schools, students were 
randomly placed in one of two conditions: (1) CIS services plus C&C or (2) CIS-only services. 

Purpose / Objective / Research Question / Focus of Study: 

The present study evaluates the effectiveness of C&C in a randomly assigned sample of 
students who were all receiving CIS services. The research questions for the study include: Are 
there differences in attendance, academics, and behavior for CIS students who also receive C&C 
compared to students who only receive CIS services? The proposed poster will report effects of 
the intervention as implemented over the course of the 2011-2012 school year. 

Setting: 

Nine middle schools, four high schools, and one middle/high school located in a large 
urban region of the southwestern US comprised the sehing for the study. In the participating 
schools, the average number of students enrolled was 1,113 (range = 173 - 2901) and attendance 
rates ranged from 84.7% to 95.5% (mean = 92.3%). Participating schools, on average, had high 
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rates of economically disadvantaged students (mean = 86%; range = 68.9% - 94.7%), as 
measured by participation in the subsidized lunch program and a large proportion (mean = 62%; 
range = 41.7% - 100%) of students meeting criteria to be considered at-risk (e.g., single parent 
family, low income, academically below grade level). 

Population / Participants / Subjects: 

Demographic details for the 189 participating students that comprise the analytic sample 
are reported in Table 1. To be eligible for participation, students must have met eligibility criteria 
for CIS services (see Texas Education Code sections 29.081 and 33.151) and demonstrated 
absenteeism (defined by 20 or more absences during the prior school year or 2 or more absences 
during the previous month). Participating students averaged 5.06 days absent and .23 disciplinary 
referrals at baseline. About half of the participants were female (56%) and the majority of 
students were Hispanic (89%). The average age of the participants was 15.1 years and the 
majority (74%) received free or reduced-price lunch. 

Intervention / Program / Practice: 

C&C is a manualized dropout prevention intervention designed to promote students’ 
engagement in school through a targeted and individualized approach (for a full description of 
C&C, see http://checkandconnect.umn.edu and Sinclair et ah, 1998). The C&C model comprises 
two primary components. The check component involves regularly monitoring student data 
related to alterable risk indicators. The connect component involves building relationships with 
students and families and facilitating basic or intensive interventions, based on student data. The 
C&C model is delivered by an adult “monitor,” who uses a case management approach to work 
with students and their families for the duration of the intervention. The primary goal of the 
monitor is “to keep education a salient issue for the student, his or her family members, and 
teachers, and to reduce and prevent the occurrence of absenteeism, suspensions, failing grades 
and other warning signs of school withdrawal” (Sinclair et ah, 1998, p. 10). The monitor works 
with students and families on his or her caseload to promote school engagement by building and 
maintaining relationships, monitoring student data related to alterable risk indicators, and 
implementing individualized interventions with students and families based on the data. 

In this study, CIS site coordinators acted as the C&C monitors. The C&C monitors were 
school-based practitioners who had been employed with CIS for a mean of 4.38 years. The 
monitors provided C&C to up to 12 students in addition to providing CIS services to their regular 
caseload. The monitors recorded attendance, disciplinary infractions, and academic performance 
weekly on a form adapted from Sinclair et al. (1998). The data were used to provide feedback to 
students, determine students’ level of risk, and develop and implement individualized 
interventions based on risk indicators. Monitors met with students weekly to discuss progress 
and the importance of staying in school and assist students with problem solving related to 
current or ongoing issues. For students exhibiting high risk on the indicators being monitored, 
individualized interventions were developed according to student risk factors and needs. 

Research Design: 

This study used a randomized block design to examine the effectiveness of C&C on 
academic performance, behavior, and attendance with at-risk middle and high school students. 
Eligible students were randomly assigned to the treatment or control condition within each of the 
14 participating schools. In fall 2011, prior to the start of the intervention and after parent 
consent and student assent, 134 students were assigned to the treatment condition (C&C + CIS) 
and 126 were assigned to the control condition (CIS only). 
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Due to attrition, posttest data were available for 189 of the 260 students randomized to 
the treatment and eontrol groups (see Figure 1). The total (27%) and differential (13%) attrition 
rates were relatively high, although not uncommon in school-based field research with at-risk 
students in urban settings (see Wilson et al., 2011; Wilson & Lipsey, 2006). Analysis of selection 
bias and pretest equivalency of the analytic sample suggested that the analytic treatment and 
control groups were statistically balanced at pretest (see Table 1). 

(Please insert Figure 1 and Table 1 here) 

Data Collection and Analysis: 

Pre and posttest measures of attendance, grades, and disciplinary infractions were 
collected from the first and last marking period report cards respectively. Performance 
composites for all students in the dataset, using English, mathematics, science, and social studies 
grades, were generated at pretest (a = .73) and posttest (a = .82). Disciplinary infractions were 
measured by the number of office referrals at pre and posttest and attendance was measured by 
the number of days absent at pre and posttest. 

Hierarchical Linear Modeling (HEM) controlled for the school-level random effects. All 
models included pretest functioning and relevant student and school covariates. Student-level 
covariates included free and reduced lunch status, race, gender, age, grade, and family income. 
School-level covariates included school size and the percentage of students considered at-risk, 
highly mobile, disadvantaged, of limited English proficiency, and the percent of students 
meeting state achievement standards. A stepwise HEM model estimation procedure was used to 
fit each dependent variable (Raudenbush et ah, 2002). In step 1, an unconditional, fixed effects 
model was fit to each outcome to estimate the intraclass correlation — or the proportion of 
variance in outcomes attributed to school-level effects. The intraclass correlation (ICC) for 
academic performance {p = .09), discipline (p = .15), and attendance (p = .12) indicated that 9%, 
15%, and 12% of the variation in posttest outcomes were attributed to school-level effects, 
respectively — sufficient magnitudes to warrant the use of hierarchical linear modeling. In step 2, 
conditional models were fit to test the intervention effects. In step 3, random slope and intercept 
models were tested by fitting cross-level interactions and random effects. Random effects and 
interaction terms were retained in the models only if significant. All associations were assessed 
by using a two-tailed test and a = .05. Model estimates were generated in STATA 1 1.0 
(StataCorp, 2005), using xtmixed and a maximum likelihood estimator — ^which performs well 
when the number of level 2 units is not large (Bryk & Raudenbush, 1992). 

Eollowing model estimation, effect sizes were estimated using approaches suggested for 
multilevel data for all outcomes significantly associated with C&C (Cohen, 1988; Spybrook, 
Raudenbush, Congdon, & Martinez, 2009). Eigure 3 represents the effect size equation, where P 
is the multilevel coefficient for the treatment variable on the outcome of interest, is the 
residual variance between schools, and represents the residual variance within schools. 

Findings / Results: 

The multilevel model estimates are offered in Table 2. Controlling for pretest 
performance and all relevant student and school level characteristics, significant differences at 
posttest were observed favoring treatment participants compared to control students on grades 
ifi=\.5Al ip = .043, 95% Cl [.047-3.048]) and disciplinary infractions ()5-.363,p=.036, 95% Cl [- 
0.70-0.02]). Effect sizes for attendance {d = 0.07) and disciplinary infractions {d = 0.27) were 
small to moderate. Contrary to prior studies, no significant differences for attendance were 
observed {S = -0.01). Using the improvement index as suggested by the Institute of Education 


SREE Spring 2014 Conference Abstract Template 


3 



Effects of Check & Connect 


Sciences (2008), these effect sizes translate into a 3% percentile improvement in grades, an 1 1% 
percent reduction in disciplinary referrals and a 0.04% percent improvement in attendance for the 
average student in the intervention compared to one in the control condition. 

(Please insert Table 2 here) 

Conclusions: 

The present study examined the effects of C&C with a diverse sample of students at high 
risk for dropout as defined by high rates of absenteeism, poor school behavior, and below- 
average academic performance — three critical risk factors in predicting eventual dropout. In 
several specific ways the current study extends the existing evidence for C&C. First, the present 
study is one of the first independently evaluated, randomized effectiveness trials of C&C. 

Second, the sample of student participants in the present study consisted largely of Hispanic 
students. Prior studies of C&C have not specifically examined the effects of the program for this 
subgroup of students who appear to be at increased risk for dropout compared to other non-white 
students (Chapman et ah, 2010). Lastly, though the findings are mixed and the effects are 
relatively small, the estimates derived in this study resulted from an intent-to-treat analysis of a 
well-defined intervention using an effectiveness approach to examine the impact of the 
intervention under real world conditions. Findings from such studies suggest schools can feasibly 
implement and use C&C to improve outcomes for at-risk students. Moreover, this randomized, 
field-based trial demonstrates that local university-school-community partnerships may 
contribute to the scientific evidence-base by evaluating interventions in real-world settings. 

As with all studies, the present study has important limitations that condition the 
generalizability of the findings. First, total and differential attrition rates exceeded acceptable 
attrition rates established by What Works Clearinghouse (WWC) to “meet evidence standards.” 
As such, the WWC guidelines for randomized controlled trials with high attrition rates were 
followed in that we examined baseline equivalence on student characteristics and outcome 
variables and controlled for demographics and pretest scores in all analytical models. Moreover, 
the analytic sample was balanced on observed variables at pretest. Second, students received 
C&C over a relatively short period — six months rather than the 2 years recommended by Sinclair 
et al. (1998). As a result, students may not have received the full benefits conferred by extended 
exposure to C&C, possibly explaining the smaller effect sizes observed. Our choice of 
outcomes — school-reported grades, office disciplinary referrals, and attendance — was both 
pragmatic as well as purposeful. That is, schools and the CIS affiliate were interested in 
positively affecting these proximal outcomes. Outcomes can be specific to school settings and 
may not always be generalizable across settings even though these data are commonly used in 
studies reporting the effects of school-based interventions (Irvin, Tobin, Sprague, Sugai, & 
Vincent, 2004). Despite these obvious limitations, randomization of students within schools 
boosts internal study validity and reduces bias resulting from different practices across schools 
settings. 

This randomized study of C&C contributes empirical evidence in support of C&C with a 
different sample of at-risk students from prior studies of C&C — Hispanic and absentee 
students — and adds a level of internal validity rarely found in studies of school-based 
intervention research. Moreover, this study provides evidence of the effects of C&C 
implemented in a real-world setting by school-based practitioners, situating effect sizes within 
the context of C&C being implemented under conditions that practitioners would normally 
experience. 
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Appendix B, Tables and Figures 

Table 1 


Demographic Characteristics and Pretest Equivalence of Treatment and Control Groups 



Total 
(N= 189) 

C&C 
(« = 89) 


Control 
(« = 100) 



Variable 

% 

N 

% 

n 

% 

n 

X\df) 

P 

Sex 









Male 

44% 

84 

38% 

34 

50% 

50 

2.66 (1) 

.10 

Female 

56% 

105 

62% 

55 

50% 

50 



Grade 









6th 

20% 

37 

19% 

17 

20% 

20 



7th 

15% 

28 

14% 

13 

15% 

15 



8th 

21% 

40 

24% 

21 

19% 

19 



9th 

23% 

43 

45% 

19 

24% 

24 

1.71 (6) 

.10 

10th 

7% 

13 

21% 

5 

8% 

8 



11th 

6% 

12 

8% 

7 

5% 

5 



12th 

8% 

16 

8% 

7 

9% 

9 



Ethnicity 









African American 

11% 

20 

15% 

13 

7% 

7 

2.88 (1) 

.09 

Hispanic 

89% 

169 

85% 

76 

93% 

93 



Free or reduced lunch 









Yes 

74% 

140 

72% 

64 

76% 

76 

0.41 (1) 

.52 

No 

26% 

49 

28% 

25 

24% 

24 




M 

SD 

M 

SD 

M 

SD 

ii' 

, 

00 

P 

Age 

15.1 

2.02 

15.2 

0.20 

15.0 

0.21 

0.51 

.61 

Grade 

8.35 

1.80 

8.34 

0.19 

8.36 

0.18 

0.04 

.97 

Income 

42.46 

36.69 

43.03 

3.95 

41.35 

3.63 

-0.20 

.84 

Academic 

performance 

75.63 

8.70 

76.56 

7.73 

74.78 

9.46 

1.36 

.17 

Discipline 

0.23 

0.65 

0.169 

0.65 

0.30 

0.72 

1.38 

.17 

Attendance 

5.06 

4.47 

4.63 

3.83 

5.44 

4.97 

1.25 

.22 
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Table 2 


Student-School Fitted Flierarchical Linear Models: The Effects of C&C (N = 189) 




Academic performance 

Discipline 


Attendance 

Level 

Effect 

h 

SE 

h 

SE 

h 

SE 

Student 

Conditional mean 

14.025*** 

7.262 

2.019** 

1.305 

13.537* 

7.776 


Pretest 


.052 

1.208*** 

.138 


.123 


Age 

-1.447** 

.576 

.076 

.127 

.760 

.761 


Grade 

1.418* 

.738 

-.188 

.154 

-1.310 

.907 


Sex 

.142 

.829 

-.149 

.183 

-.796 

1.093 


Free lunch 

-2.773** 

1.125 

-.070 

.235 

.551 

1.392 


Income 

.014 

.011 

-.003 

.003 

-.026* 

.015 


African American 

1.072 

1.400 

.544* 

.139 

-1.736 

1.905 


Tx (C&C) 

1.547* 

.765 

-.363* 

.173 

-.577 

1.033 

School 

School size 

.002 

.003 

-.000 

.000 

.002 

.002 


% disadvantaged 

.017 

.263 

-.009 

.039 

.235 

.224 


% LEP 

.415 

.255 

.035 

.036 

-.189 

.202 


% at risk 

-.171 

.113 

-.007 

.016 

.013 

.089 


% mobility 


.155 

.021 

.021 

.062 

.117 


% average 
performance 

.079 

.112 

.017 

.017 

.015 

.095 


Note. All hypothesis tests are two tailed. Aeademie performanee = eomposite of English, 
mathematics, science, and social studies; discipline = total number of office referrals; 
attendance = total number of days missed. Tx = C&C = 1, control = 0; income, and 
percentage of disadvantaged students, LEP (limited English proficiency), at risk, mobility, 
and average school-level performance all grand mean centered. 

*/> < .10. **/> < .05. ***/>< .001. 
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§ 


C&C + CIS 


CIS only 



(treatment group) 


(control group) 

.a 


n = 134; 52% 


«= 126; 48% 

<c 


J=14; 100% 


J=14; 100% 



Analytic sample 


Analytic sample 

n = 89; 66% 


n = 100; 79% 

J=14; 100% 


J=14; 100% 


Figure 1. Participant flow chart. Participant percentages reported at eaeh stage are 
ealeulated by using number of partieipants in prior stage. N{n) = number of students; J(/) 
= number of sehools. 
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